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Summary. 

Almost  nothing  can  be  deduced  about  a general  S-D  surface  given  only  its 
occluding  contours  in  an  image,  yet  contour  information  is  easily  and  effectively  used  by  us 
to  iffer  the  shape  of  a surface.  Therefore,  implicit  in  the  perceptual  analysis  of  occluding 
contour  must  Me  various  assumptions  about  the  viewed  surfaces.  The  assumptions  that  seem 
mom  natural  are  (a)  that  the  distinction  between  convex  and  concave  segments  ref  leas  real 
properties  of  the  viewed  surface;  and  (b)  that  contiguous  portions  of  contour  arise  from 
contiguous  parts  of  the  viewed  surface  - i.t,  there  are  no  invisible  obscuring  edges.  It  is 
proved  that,  for  smooth  surfaces,  these  assumptions  are  essentially  equivalent  to  assuming 
that  the  viewed  surface  is  a generalised  cone  Methods  are  defined  for  finding  the  axis  of 
such  a cone,  and  for  segmenting  a surface  constructed  of  several  cones  into  its  components, 
whose  axes  can  then  be  found  separately.  These  methods,  together  with  the  algorithms  for 
implementing  them  devised  by  Vatan  k Marr  (1977),  provide  one  link  between  an 
uninterpreted  figure  extracted  from  an  imago,  and  the  S-D  representation  theory  of  Marr  It 
Nishihara  (1977). 
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Introduction 

When  we  look  at  the  silhouettes  in  Picasso’s  work  "Rites  of  Spring”  (figure  I),  we 
perceive  them  in  terms  of  very  particular  S-D  shapes,  some  familiar,  some  less  so.  This  is 
quite  remarkable,  because  the  silhouettes  could  in  theory  have  been  generated  by  an  infinite 
variety  of  shapes  which,  from  other  viewpoints,  have  no  discernable  similarities  to  the 
shapes  we  perceive.  One  can  perhaps  attribute  part  of  the  phenomenon  to  a familiarity 
with  the  depicted  shapes;  but  not  all  of  it,  because  one  can  use  the  medium  of  a silhouette 
to  convey  a new  shape,  and  because  even  with  considerable  effort  it  is  difficult  to  Imagine 
the  more  bizarre  three-dimensional  surfaces  that  could  have  given  rise  to  the  same 
silhouettes. 

This  phenomenon  is  of  quite  general  importance  for  the  analysis  of  an  image. 
The  boundary  of  a silhouette  is  simply  one  type  of  occluding  contour  (see  e.g.  Waltz  1975), 
and  such  contours  are  an  artist's  principal  means  of  conveying  information  about  shape. 
The  paradox  is  that  they  apparently  tell  us  more  about  shape  than  they  should.  For 
example,  neighbouring  points  on  such  a contour  could  in  general  arise  from  widely 
separated  points  on  the  original  surface,  but  our  perceptual  interpretation  usually  ignores 
this  possibility. 

This  means  that  implicit  in  the  way  we  interpret  an  occluding  contour,  there  must 
lie  some  a priori  assumptions  that  allow  us  to  infer  a shape  from  an  outline.  If  a surface 
violates  these  assumptions,  our  analysis  will  be  wrong,  in  the  tense  that  the  shape  we  assign 
to  the  contours  will  differ  from  the  shape  that  actually  caused  them.  An  everyday  example 
of  this  phenomenon  is  the  shadowgraph,  where  the  appropriate  arrangement  of  one’s  hands 
can,  to  the  surprise  and  delight  of  a child,  produce  the  shadow  of  an  apparently  quite 
different  shape,  like  a duck  or  a rabbit 

What  assumptions  is  it  reasonable  to  suppose  that  we  make?  I shall  argue  for 
these  two;  (a)  that  nearby  points  on  a contour  correspond  to  nearby  points  on  the  viewed 
surface;  and  (b)  that  the  distinction  between  convexities  and  concavities  in  a contour 
reflects  real  properties  of  the  surface,  not  an  artifact  of  perspective. 

Some  surfaces  seen  from  some  viewpoints  will  satisfy  these  conditions,  and  some 
will  not.  Our  first  task  is  to  understand  what  it  is  about  a surface  that  makes  it  satisf y these 
assumptions,  and  the  main  result  of  the  first  part  of  the  paper  achieves  this.  Theorem  I 
shows  that,  if  the  assumptions  (a)  and  (b)  hold  for  all  distant  vantage  points  such  that  the 
line  of  sight  lies  parallel  to  some  fixed  plane,  then  the  viewed  surface  must  be  a generalized 
cone.  (A  generalized  cone  is  the  surface  swept  out  by  moving  a cross-section  of  fixed  shape 
but  smoothly  varying  size,  along  an  axis,  as  illustrated  in  figure  5). 

This  result  is  strong  and  surprising.  It  means  that  if  one  has  a method  for 
interpreting  contours  that  relies  on  assumptions  (a)  and  (b),  then  the  method  implicitly 
assumes  that  the  viewed  shape  is  a generalized  cone  One  can  think  of  such  a method  as 
first  throwing  a generalized  cone  blanket  round  the  viewed  shape,  and  then  describing  the 
shape  of  the  blanket.  This  in  turn  means  that  the  representation  of  S-D  shape  that  is 
subsequently  used  can,  without  further  loss  of  information,  be  based  on  generalized  cones 
(like  that  of  Marr  k Nishihara  1977). 
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l Rites  of  spring"  by  Picasso.  W«  immediately  interpret  the  silhouettes  in  terms  of 
particular  3-D  surfaces,  despite  the  paucity  of  information  in  the  image.  In  order  to  do  this. 
we  must  be  bringing  additional  assumptions  and  constraints  to  bear  on  the  analysis  of  these 
contours'  shapes.  This  article  enquires  about  the  nature  of  this  a priori  information. 


DOCK 


1 The  three-dimensional  surface  2.  viewed  from  a point  V,  form*  the  silhouette  Sy  in  the 
imago  » to  the  imaging  proem  a The  boundary  of  $y.  obtained  by  the  boundary  operator  d 
U denoted  by  Cy  and  we  call  it  the  contour  of  2.  The  set  of  points  on  2 that » maps  onto 
Cy  we  ad  the  contour  generator  of  Cy,  and  it  is  denoted  by  Ty.  The  map  from  2 to  Ty 
induced  by  3 U denoted  by  l 
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the  contour  Cy. 

This  means  that  each  line  of  sight  from  V to  the  edge  of  Z - that  is,  from  V to  Ty  — 
touches  Z at  only  one  point,  not  at  two  points  (as  shown  in  figure  20)  or  along  a line 
segment  The  condition  that  each  line  of  sight  touches  Z at  one,  rather  than  at  turo  or  a 
finite  number  of  points,  is  equivalent  to  saying  that  Z it  convex,  as  seen  from  this  viewpoint 
(see  figure  20).  This  is  not  as  strong  a condition  as  It  appears  at  first  sight,  because  in 
practise  It  will  not  usually  be  imposed  for  all  viewpoints  (eg.  theorem  I),  and  there  are  ways 
of  regaining  the  generality  that  it  excludes  (theorem  5).  Forbidding  the  line  of  sight  from 
touching  Z along  a line  segment  (as  can  happen  for  example  If  one  views  a cube  from  a 
direction  parallel  with  one  of  its  faces)  is  only  a technical  restriction;  one  can  escape  it 
without  changing  the  situation  in  an  important  way  by  deforming  the  viewed  surface  very 
slightly. 

Restriction  Ri:  Nearby  points  on  the  contour  Cy  arise  from  nearby  points  on  the  contour 
generator  Ty- 

This  condition  is  a powerful  one,  and  it  best  explained  by  figure  S.  Suppose  that  the 
contour  ab  of  figure  S really  arose  from  two  hills,  but  the  dotted  portion  of  b happened  to  be 
invisible.  Then  the  contour  generator  of  ab  would  be  discontinuous  at  P,  where  it  leaps 
from  one  hill  to  the  next.  This  is  the  situation  that  Ri  forbids,  and  it  is  essentially 
equivalent  to  assuming  that  the  image  contains  no  invisible  obscuring  edges.  Together,  R2 
and  R)  imply  that  the  contour  generator  Vy  Is  a continuous  curve  across  Z - i-e.  that  it 
does  not  Jump  erratically  from  place  to  place  on  Z.  Ri  is  a strong  condition,  but  without  it 
one  can  say  almost  nothing  about  Z and  I know  of  no  way  to  proceed  without  it. 
Fortunately  it  is  obeyed  by  most  real-world  images. 

Remark 

In  fact,  R2  and  Ri  are  not  quite  independent,  since  if  one  assumes  that  the 
surface  Z is  bounded,  R3  is  a consequence  of  R2.  To  see  this,  notice  that  at  points  like  P in 
figure  S where  Ri  is  violated,  the  viewing  ray  to  P grates  both  hills,  and  so  causes  a 
violation  of  R2.  Nevertheless,  the  two  restrictions  have  sufficiently  different  meanings  to 
make  it  worth  stating  them  separately. 

Using  points  of  inflexion 

The  restrictions  Rl-Ri  are  very  general,  and  guarantee  only  the  integrity  of  Cy 
and  Ty,  not  their  interpretabiUty.  Let  us  therefore  suppose  that  a contour  Cy,  like  that 
shown  In  figure  4,  was  Obtained  under  conditions  that  satisfy  these  restrictions,  and  enquire 
what  properties  of  Cy  we  can  rely  upon.  Ckarly,  no  metrical  properties  of  Cy  can  be  used, 
because  Cy  arises  from  viewing  a surface  Z at  an  unknown  orientation  - i t.  through  at 
best  a linear  operator,  and  such  operators  do  not  preserve  distances.  The  values  of  Cyi 
maxima  and  minima,  and  their  separation,  remain  uninformative  until  substantially  more  is 
known  about  Z and  the  perspective  from  which  it  is  being  viewed.  But  the  qualitative 


i.  Restriction  R),  which  states  that  nearby  points  on  a contour  arise  from  nearby  points  on 
iu  contour  generator,  is  essentially  equivalent  to  assuming  that  there  are  no  hidden 
obscuring  edges.  If  a and  b are  the  outlines  of  two  hills,  and  if  the  dotted  portion  of  b were 
invisible,  R3  would  be  violated  at  the  point  P where  the  contour  generator  leaps  from  one 
hill  to  the  other. 


Analysis  of  Contour 


9 


Man- 


notions  of  maxima  and  minima  on  a planar  curve  are  preserved  by  a linear  operator  - that 
is,  the  distinction  between  convex  and  conave  is  invariant  This  fact  is  aptured  by  Lemma 
I of  the  appendix. 

Let  us  therefore  suppose  that  we  have  been  presented  with  a contour  segment  like 
that  shown  in  figure  4.  Restriction  R)  guarantees  that  adjacent  points  on  the  contour  arise 
from  adjacent  points  on  the  surface  Z,  but  no  metrical  features  are  yet  reliable.  The  only 
straightforward  feature  that  remains  is  the  distinction  between  a convex  contour  segment 
and  a conave  one,  which  rests  in  turn  on  the  notion  of  an  inflexion  point  For  a general 
surface  Z end  contour  generator  Ty,  even  points  of  inflexion  in  Cy  wilt  often  be 
meaningless,  and  to  attribute  signiflana  to  them  is  to  make  an  additional  assumption  about 
Z.  So  we  next  ask,  how  exactly  should  we  formulate  the  assumption  that  points  of  inflexion 
are  significant? 

The  restrictions  RI-R3  allow  us  to  think  of  Ty.  the  contour  generator  of  Cy  on  Z. 
as  a smooth  piece  of  wire  bent  in  S-space.  Ter  ' ex  ion  points  on  Cy  to  be  signifiant 
however,  lemma  I (see  the  appendix)  tells  us  that  we  need  two  things;  (1)  the  transformation 
due  to  the  imaging  process  that  produces  Cy  must  be  linear,  and  (2)  the  curve  on  which  that 
transform  acts  mutt  lie  in  a plane.  Beause  the  general  perspective  transformation  is  not 
linear,  condition  (I)  tells  us  that  our  whole  theory  applies  only  to  distant  viewing  points, 
beause  only  in  these  conditions  is  the  imaging  process  a linear  projection.  Condition  (2) 
informs  us  that  the  convex-conave  distinction  an  be  meaningful  in  Cy  only  if  the  bent 
wire  that  is  Ty  lies  in  a plane.  This  gives  us  our  fourth  condition. 

Restriction  R4:  T he  contour  generator  Ty  of  Cy  is  planar. 

This  condition  is  a strong  one,  and  sharply  delimits  the  claw  of  admissible  surfaces  Z.  There 
seems  however  to  be  no  way  of  avoiding  it  if  one  wishes  to  use  the  distinction  between 
convex  and  conave  contour  segments. 


Implications  of  the  four  rostriotions 

A generalized  cone,  illustrated  in  figure  5,  is  defined  to  be  the  surface  swept  out  by 
moving  a simple  smooth  cross-section  along  some  axis,  at  the  same  time  magnifying  or 
contracting  it  in  a smoothly  varying  way.  This  cross-section  is  defined  by  the  function 
p(r,  9)  • o,  and  when  the  cross-section  is  convex,  we  shall  use  cylindrkal  coordinates  r - 
p(9).  The  magnifiation  of  the  cross-section  at  each  point  is  specified  by  the  function  h(t), 
where  z is  the  distance  measured  along  the  cone's  axis.  The  axis  itself  will  be  labelled  A. 
Notice  that  in  general  the  z axis  need  not  be  perpendicular  to  the  plane  z • 0 of  the  cross- 
section.  These  conventions  are  illustrated  by  figure  5. 

We  may  demand  that  the  restrictions  R2  • R4  hold  for  all  views,  or  for  a subclass 
of  the  possible  views  of  Z.  If  we  demand  that  they  hold  for  only  one  (distant)  viewpoint, 
this  imposes  no  interesting  restrictions  on  the  nature  of  Z.  Theorem  1 studies  the  two 
dimensional  case,  when  the  restrictions  are  assumed  to  hold  for  all  distant  viewpoints  whose 


&.  T»»#  definition  of  a generalised  com.  In  this  article,  a |enerallied  cone  to  the  surface 
generated  toy  moving  a smooth  cross-section  p along  a itraif  ht  axis  A-  The  cross-section  may 
vary  smoothly  m stoe  (as  prescribed  by  the  axial  scaling  function  Hi)),  but  its  shape  remains 
constant  The  eccentricity  of  the  com  to  the  angle  + between  its  axis  and  a plans  containing 
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line* of  sight  lie  parallel  to  some  fixed  plane,  and  it  is  the  most  interesting  result  of  the 
section.  Finally,  theorem  2 studies  the  consequences  of  assuming  that  the  restrictions  hold 
for  att  distant  viewpoints. 


7* h basic  therms 

Earlier,  we  defined  Sy  to  be  the  image  of  Z as  seen  from  the  vantage  point  V 
(figure  2).  This  is  equivalent  to  saying  that  Sy  is  the  perspective  projection  of  Z from  the 
point  V.  In  theorem  I,  we  shall  make  two  simplifying  assumptions  about  the  projection  Sy. 
first  that  the  projection  is  orthogonal  which  is  approximated  when  the  vantage  point  V is 
very  distant  from  Z compared  to  its  siie,  and  second,  that  the  viewing  directions  are 
confined  to  a plane  II  round  Z and  which  intersects  Z.  We  deal  in  some  sense  only  with 
"side*  views  of  Z,  and  are  forbidding  "end-on"  views.  Such  projections  are  completely 
specified  by  the  direction  of  the  vantage  point  from  Z in  the  confining  plane  n,  and  we 
denote  this  by  the  angle  +.  We  shall  use  the  notation  S^,  and  in  place  of  Sy,  Cy 
and  IV  to  indicate  that  the  above  restrictions  are  in  effect.  The  proofs  of  theorems  I and  2 
are  set  out  in  the  appendix.  I give  here  their  statements  in  plain  English. 

T harm  I.  Z is  a generalized  cone  with  convex  cross-section  if  and  only  if 
fU  is  satisfied,  and  R2  • R4  are  satisfied  for  all  orthogonal  projections  d 
associated  with  some  plane  II.  in  the  sense  defined  above  This  plane  lies 
parallel  to  the  cross-section  of  the  cone 

Therm  2.  Z satisfies  R1  and  R2  - R4  for  all  distant  vantage  points  V if 
and  only  If  Z is  a quadratic  surface 


Remarks  about  theorems  1 and  2 

It  is  theorem  1 that  allows  the  crucial  step  for  the  overall  argument  It  says  that  if, 
for  distant  viewpoints  whoa*  viewing  directions  lie  parallel  to  some  plane  a surface’s  shape 
can  successfully  be  inferred  using  only  the  convexities  and  concavities  of  its  bounding 
contours  in  an  image  then  that  surface  is  a generalized  cone  with  convex  cross-section,  or  is 
composed  of  them.  Hence  if  one  assumes  that  one  can  discover  a surface’s  shape  from  such 
information,  then  this  is  equivalent  to  assuming  that  the  viewed  surface  is  a generalized 
cone  The  assumption  of  theorem  i,  about  orthogonal  projections  parallel  to  the  plane  of 
the  cross-section,  is  tolerable  because  as  we  shall  see  the  methods  to  which  the  theory  gives 
rise  usually  degrade  only  slowly  as  one  moves  nearer,  increasing  the  effects  of  perspective;  or 
out  of  the  plane  of  the  crow-section.  Furthermore  there  does  appear  to  be  something  special 
about  the  perception  of  views  that  look  down  the  z-axls  of  the  figure  (see  the  remarks  made 
by  Marr  Sc  Nishihara  (1477)  about  Warrington  Sc  Taylor’s  (I97S)  "unconventional  views"). 

Theorem  2 is  interesting,  because  it  shows  how  very  strong  our  restrictions  are. 
One  can  gain  a feel  for  how  the  ptanpr  condition  R4  fails  for  higher-order  surfaces  by 
studying  the  behaviour  of  ***  ♦ y**  ■ / (see  figure  6).  This  surface  is  a sphere  for  n 

• /,  and  tends  to  a cube  as  n grows  large.  The  contour  generator  Ty.  which  is  a circle  for  n 
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• / (figure  6a),  becomes  the  outline  marked  with  thick  lines  in  figure  6b  for  high  values  of 
n.  This  contour-generator  is  clearly  not  planar,  as  n increases,  the  lower  third  of  the  contour 
generator  Is  pulled  towards  the  viewer,  and  the  upper  third  is  pushed  away. 

These  results  provide  a further  argument  for  using  something  based  on 
generalised  cylinders  (Binford  1971)  for  the  internal  representation  of  shape  (see  Marr  4c 
Nishihara  1977),  an  argument  based  not  on  utility,  as  most  other  justifications  are,  but  on  the 
assumptions  implicit  in  the  decoding  of  an  image.  It  is  indeed  extremely  fortunate  that 
many  important  three-dimensional  structures  can  be  closely  approximated  by  a few 
generalized  cones,  although  it  is  not  accidental  that  objects  whose  shape  teas  achieved  by 
growth  like  limbs  and  stalagmites,  can  be  so  approximated. 


2:  Interpreting  the  Image  of  a single  generalised  oone 

Theorem  I essentially  tells  us  that,  when  trying  to  infer  the  shape  of  a surface 
from  its  bounding  contours  in  an  image  we  cannot  avoid  assuming  that  the  surface  is  a 
generalized  cone.  We  are  now  faced  with  an  obvious  question.  If  we  assume  that  our  data 
consists  of  contours  in  the  image  of  a generalized  cone,  how  may  we  interpret  them?  To 
specify  a generalized  cone,  we  have  to  specify  its  axis  A,  cross-section  p($),  and  axial  scaling 
function  Afz>,  how  can  we  discover  them  from  an  image? 

The  answer  to  this  question  commences  with  theorem  S,  which  shows  how  the 
occluding  contours  in  an  image  may  be  used  to  find  the  "image"  of  the  cone’s  axis  for  those 
distant  viewpoints  that  lie  in  the  privileged  viewing  plane  referred  to  in  theorem  I.  In 
general,  of  course,  our  viewpoint  will  not  lie  in  this  plane,  and  so  we  have  to  examine  the 
stability  of  this  result  as  the  viewpoint  moves  out  of  the  plane.  This  is  achieved  by  theorem 
4,  which  introduces  a new  concept  called  the  skeleton  of  a generalized  cone.  The  skeleton  is 
not  a difficult  idea,  however,  since  it  is  very  like  the  set  of  lines  a cartoonist  draws  to  convey 
the  shape  of  a curved  object.  The  idea  of  a skeleton  allows  us  to  extend  the  theory  to 
generalized  cones  whose  crow-section  is  not  convex.  Requiring  the  restriction  R2  to  hold  for 
all  ^-projections  essentially  forbids  this  daw  of  cones,  and  I said  earlier  that  one  can 
circumvent  this  restriction  in  practise.  Theorem  5 shows  how.  Finally,  there  is  a short 
discussion  about  cases  in  which  the  cone  is  viewed  from  a nearby  rather  than  from  a distant 
point,  and  cases  in  which  the  axis  of  the  cone  is  not  a straight  line. 

The  overall  purpose  of  the  section  is  to  give  a set  of  methods  for  interpreting  the 
image  of  a single  generalized  cone  The  methods  derived  here  will  not  succeed  for  all  views; 
they  will  fail  when  the  image  of  the  cone’s  axis  is  substantially  foreshortened.  It  is  part  of 
the  overall  theory  that  such  views  have  to  be  handled  differently  (Marr  It  Nishihara  1977). 

Finding  Ike  axis  from  a favourable  view 

Provided  that  the  viewed  surface  is  a generalized  cone,  and  that  the  viewing  point 
satisfies  the  conditions  of  theorem  1,  the  axis  of  the  cone  may  easily  be  determined  by  the 
rough  symmetry  formed  around  it 

Theorem  3 (Axial  Symmetry).  Let  Z be  a generalized  cone  with  convex  crow- 


6.  The  surface  ***  ♦ y*n  ♦ s*"  • / f or  a • 1 (figure  6a)  and  n • 1000  (figure  6b).  The 
contour  generator  is  shown  in  thick  lines.  It  is  planar  (a  circle)  in  6a,  but  not  for  n > /. 
Figure  6b  shows  how  the  contour  generator  is  pulled  out  of  a plane  for  high  values  of  n. 
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section  p(9),  and  let  the  cross-section  scaling  function  A(z)  contain  at  least 
one  concavity.  Then  for  all  viewing  directions  4 

(i)  the  silhouette  of  2,  C^.  decomposes  into  n > 2 contour  segments  by 
splitting  it  at  points  of  inflexion; 

(ii)  the  image  of  the  axis  A of  2 establishes  an  axial  symmetry  between  at 
least  (n  - 2)  contour  segments,  including  all  conave  ones.  Corresponding 
segments  are  either  both  convex  or  both  concave; 

(Ui)  the  ratios  of  the  distances  of  corresponding  segments  either  side  of  the 
axis  of  symmetry  are  all  the  same. 

Corollary:  The  image  of  the  axis  of  2 is  uniquely  determined  if  there  exists  only  one  such 
axial  symmetty. 

This  theorem  is  best  explained  by  looking  at  figure  7.  Here,  we  see  that  the 
contour  divides  at  inflexion  points  into  three  segments,  labelled  Cj  to  Cy  The  two  conave 
segments  C;  and  C}  are  roughly  symmetric  about  the  image  of  the  cone’s  axis  A,  although 
their  distances  away  from  the  axis  may  not  be  equal  The  third  clause  of  the  theorem  states 
that,  if  Cj  is  half  as  far  from  A as  Cy,  then  the  same  will  be  true  of  all  other  segments  that 
correspond  under  the  symmetry.  We  shall  all  the  type  of  symmetry  established  by  theorem 
S a qualitative  symmetry.  Its  important  features  are  (a)  that  it  holds  between  convex  or 
conave  segments  of  a contour,  and  (b)  that  it  includes  a sating  factor. 

Although  there  is  a point-wise  symmetry  between  the  contour  on  the  two  sides  of 
the  axis,  unless  the  viewed  surface  is  a right  generalised  cone  and  the  contours  are 
faithfully  diagnosed  in  the  image,  such  symmetries  are  expensive  to  detect.  A qualitative 
symmetry,  on  the  other  hand,  does  not  have  to  be  found  on  a point-by-point  basis.  This  is 
important  because  it  makes  finding  the  symmetry,  and  hence  its  axis,  a practical 
computational  proposition.  By  dividing  the  contour  into  convex  and  conave  segments  and 
noticing  that  the  symmetry  preserves  this  distinction,  we  have  greatly  reduced  the  number  of 
items  that  have  to  be  examined  and  made  the  computational  load  acceptable. 

There  is  one  other  point  of  importance  about  this  result  and  it  comes  from  the 
corollary,  which  says  that  if  only  one  symmetry  exists  among  the  contours,  the  axis  of  Z is 
determined  uniquely.  This  mans  that  the  analysis  of  contour  is  self-checking,  and  one  does 
not  have  to  appal  to  the  "familiarity*  of  the  deduced  shape  to  know  that  one  has  a valid 
interpretation  erf  the  image.  This  is  of  course  essential  if  one  is  to  be  able  to  analyte  novel 
shapes.  The  reader  will  observe  that  all  of  the  theorems,  that  are  directed  at  the  analysis  of 
contour,  have  uniqueness  corollaries  like  that  of  theorem  S.  It  is  on  these  that  the  algorithms 
themselves  will  rest  most  directly. 

V lowing  directions  not  coptoner  with  a cone’s  cross-section 

We  next  ask  what  happens  to  the  generalized  cones  of  theorem  S if  the  viewpoint 
remains  distant,  but  if  the  viewing  direction  moves  out  of  its  constraining  plane  II?  As  long 
as  the  image  Sy  approximates  an  orthogonal  projection  parallel  to  the  plane  of  the 
generalised  cone’s  cross-section,  variations  in  the  silhouette  of  2 are  due  to  changes  in  the 
saling  function  k(s)  along  the  cone’s  axis  A.  as  illustrated  by  figure  8a.  On  the  other  hand. 
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when  the  viewpoint  is  moved  so  drastically  that  the  viewing  direction  lies  parallel  to  A.  the 
silhouette  of  2 is  due  entirely  to  p(0),  its  cross-section  function  (figure  8b),  and  is  in  fact  due 
to  the  cross-section  of  Z at  the  point  where  h(z)  achieves  its  maximum  value. 

At  both  of  these  extremes  the  contour  generator  Ty  of  Z is  planar,  but  for  other 
viewing  directions  it  need  not  be.  This  is  obviously  true  for  a dumbell  shape,  or  for  an  ice- 
cream cone,  but  theorem  2 assures  is  that  it  is  more  insidiously  true  even  for  a surface  as 
simple  as  an  egg,  where  the  contour  generator  is  a circle  for  the  end-view,  a near  oval  for  a 
side  view,  and  slides  from  one  to  the  other  in  between  (see  figure  10b  below).  Another 
example  is  the  surface  shown  in  figure  8c;  the  contour  generator  from  this  view  is  clearly 
not  planar. 

How  are  we  to  handle  such  views?  For  convex  objects  like  an  egg,  where  the  only 
occluding  contours  arise  from  its  silhouette,  there  is  very  little  more  one  can  do  to  infer  its 
shape  when  seen  from  one  of  the  intermediate  views,  unless  one  knows  something  about  the 
orientation  of  the  egg  relative  to  the  viewer.  For  objects  that  are  not  convex,  like  the  double 
spike  of  figure  8,  one  can  separate  the  contours  that  arise  in  the  image  into  two  classes; 
those  approximately  due  to  the  "sides"  of  the  figure  (the  two  spikes  separated  in  figure  8d), 
and  those  approximately  due  to  a cross-section,  like  the  central  ellipse  in  figure  8d. 

This  division  gives  us  our  main  tool  for  analyzing  non-standard  views,  and  it  is 
best  explained  with  the  help  of  figure  9.  Suppose  that  a generalized  cone  Z is  being  viewed 
from  a distant  point  V and  the  line  of  sight  is  not  parallel  to  the  plane  of  the  cone’s  cross- 
section.  The  contour  generator  for  viewpoint  V is  approximated  by  two  components.  One 
is  easy  to  define;  it  is  the  places  on  Z where  the  size  of  the  cross-section  is  stationary  - that 
is,  where  A (z)  achieves  a maximum  or  minimum.  For  an  egg.  it  is  the  fattest  cross-section, 
and  other  examples  are  shown  shaded  in  figure  10.  We  call  these  curves  radial  extremities, 
and  denote  them  by  T - notice  there  is  no  suffix,  since  T does  not  depend  on  the  vantage 
point.  The  other  idea  we  have  to  make  as  precise  as  possible  is  what  we  mean  by  the  ■’sides” 
of  Z from  a viewpoint  such  as  V,  and  for  this  we  make  the  construction  illustrated  in  figure 
9.  We  drop  a perpendicular  from  V to  W,  which  does  lie  in  the  standard  viewing  plane. 
Then  the  contour  generator  for  viewpoint  V is  approximately  the  projection  of  the  contour 
generator  Tyl  for  W,  which  is  simply  for  some  angle  4-  For  example,  in  the  case  of  an 
egg  of  length  l and  diameter  d,  the  skeleton  (shown  in  figure  10b)  has  length  iMnfx)  and 
width  d,  when  viewed  in  a plane  containing  the  egg’s  major  axis  at  an  angle  x to  it  For 
angles  where  ljin(x)  > d,  this  is  a reasonable  approximation,  and  when  lJln(x)  < d.  we 
have  an  ‘unconventional”  view. 

The  reason  why  the  skeleton  is  a useful  construct  for  recognition  is  that  one  can 
detect  its  pretence  in  an  image  by  the  many  relationships  that  exist  among  its  parts.  In  fact, 
we  can  use  these  relationships  to  set  up  constraints  on  a set  of  occluding  contours  such  that 
if  those  constraints  are  alt  satisfied  by  a unique  interpretation  of  the  contours  in  the  image, 
we  can  be  reasonably  certain  that  we  have  found  a skeleton,  and  hence  can  interpret  the 
contours  as  arising  from  a generalised  cone  I whose  axis  is  then  determined.  The  relations 
themselves  consist  of  qualitative  symmetries  and  parallelism,  and  are  preserved  by  an 
orthogonal  projection.  Hence  provided  that  the  contours  as  seen  from  vantage  point  V in 
figure  9 are  approximately  the  projection  of  the  contours  as  seen  from  Vf,  the  relations  will 
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10.  Examples  of  skeletons  of  surfaces,  as  defined  In  the  (ext  and  In  figures  8 and  9.  The 
cross-sections  responsible  for  the  radial  extremities  of  the  surfaces  are  shown  shaded.  The 
skeleton  ceases  to  be  a raasonaMs  approximation  to  the  contours  that  occur  In  the  Image 
whenever  the  viewing  angle  Is  such  as  to  make  the  projection  of  the  length  of  the  cone  less 
than  the  orthogonal  projection  of  its  width.  For  such  views,  the  methods  of  this  article  will 
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still  hold  in  the  image  formed  from  V. 

Thtorem  4 (Skoloton  Thtortn).  Let  Tyl  u T be  the  skeleton  of  Z associated 
with  some  vantage  point  V.  Then  provided  that  Cy  can  be  thought  of  as 
being  formed  by  the  orthogonal  projection  of  Tyl  along  the  direction  to 
the  vantage  point  V, 

(i)  Cy  is  qualitatively  symmetric  about  the  image  of  the  axis  A of  Z,  in  the 
sense  of  theorem  3 

(ii)  the  image  of  T consists  of  one  or  more  connected  components,  through 
which  A passes,  and  between  any  two  of  which  there  exists  a mapping  that 
is  (I  - I),  continuous  and  onto,  that  preserves  the  gradient  of  the  image  of  T 
at  each  point 

Corollary:  If  Z obeys  the  prerequisites  of  theorem  4,  and  if  the  image  of  its  skeleton 
decomposes  in  a unique  way  into  two  components  that  satisfy  conditions  (i)  and  (ii)  of  the 
theorem,  then  these  components  are  the  images  of  Ty  and  of  T.  The  Mis  of  symmetry  of 
the  image  of  Tp  is  the  image  of  the  Mis  of  Z- 

This  theorem  makes  explicit  the  many  relations  between  the  elements  of  a 
skeleton’s  image,  and  its  practical  importance  is  illustrated  by  figure  11.  The  theorem  states 
that  the  image  of  the  "sides”  obeys  quite  well  the  symmetry  relation  of  theorem  3,  and  one 
can  see  from  the  figure  that  this  is  true  of  the  sides  of  the  bucket  in  the  image.  The  axis  of 
symmetry  of  the  sides  is  the  Mis  of  the  bucket  The  theorem  also  ays  that  the  images  of  a 
cone’s  radial  extremities  are  all  parallel  to  one  another,  and  embrace  the  cone’s  axis.  In 
figure  1L  there  is  a clear  parallel  relationship  between  the  image  of  the  bucket’s  top,  the 
corrugations  in  its  side,  and  the  visible  part  of  its  base. 

As  in  the  case  of  theorem  3,  the  diagnostic  power  of  this  result  lies  in  the  corollary. 
It  does  not  guarantee  that  a given  set  of  occluding  contours  can  be  interpreted,  but  if  a 
unique  interpretation  exists  that  satisfies  these  conditions,  then  it  will  be  correct.  In  a real 
image^many  parts  of  a cone’s  skeleton  will  be  obscured,  but  this  hampers  the  finding  of 
relationships  like  parallelism  and  qualitative  symmetry  only  slightly.  One  can  devise  a 
cooperative  algorithm  (Marr  It  Poggio  1976)  that  can  operate  on  the  description  of  a contour 
to  find  relationships  of  this  kind  between  its  pieces  (Vatan  6c  Marr  1977). 

GtruroUud  torus  whoso  eross-stetion  Is  not  convex 

We  are  now  ready  to  extend  the  theory  to  the  case  where  the  cone’s  cross-section 
contains  concavities.  The  important  difference  between  this  and  the  case  where  the  cross- 
section  is  convex  is  that  occluding  contours  can  now  also  arise  from  local  maxima  and 
minima  in  the  cross  section  p.  For  example,  in  the  image  of  a fluted  pillar,  there  are  many 
lines  running  parallel  to  the  Mis  of  the  pillar,  corresponding  to  the  local  maxima  or  minima 
in  the  pillar's  cross  section. 

This  gives  us  the  extra  tool  we  need  to  extend  the  analysis  of  theorem  4. 
Contours  that  are  due  to  convexities  and  concavities  in  the  cross-section  p behave  like  the 
fluting  on  a pillar,  so  we  call  them  the  cone’s  /fating  and  denote  them  by  the  letter  •.  The 


11.  Methods  based  on  the  corollary  to  theorem  4 suffice  to  solve  this  image  of  a bucket.  An 
axial  symmetry  is  established  by  its  sides  about  the  bucket’s  axis  (shown  thickened),  and  a 
parallel  relationship  holds  between  components  of  its  radial  extremity.  Here,  these  are  the 
bucket’s  top  and  bottom,  and  the  corrugations  in  its  side. 
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fluting  on  a cone  with  variable  cross-section  behaves  rather  like  the  silhouette  of  theorem  3. 
Convexities  and  concavities  in  the  fluting  on  one  side  of  the  cone’s  axis  are  in  qualitative 
symmetry  with  the  fluting  on  the  other  side  (as  seen  by  the  viewer).  This  means  that 
contours  in  the  fluting  obey  a set  of  qualitative  symmetry  and  rough  parallel  relations 
among  themselves,  similar  but  orthogonal  to  those  obeyed  by  the  radial  extremities.  These 
relationships  can  be  used  to  interpret  the  contours  in  an  image,  in  a way  analogous  to 
theorem  4.  The  extension  of  theorem  4 to  the  case  of  a cone  with  fluting  is  theorem  5. 

Theorem  5.  Let  Ty!  u T u 4 be  the  skeleton  and  fluting  of  2 associated  with 
some  distant  vantage  point  V.  Then 

(i)  Cy  and  T obey  theorem  4. 

(ii)  The  image  of  each  portion  {(h(z).p($ 4),  z),  for  fixed  #j  and  varying 

*}  of  the  fluting  is  either  a straight  line,  or  it  divides  into  convex  and 
concave  segments  that  are  in  (I  • I)  correspondence  with  the  convexities  and 
concavities  in  that  part  of  the  image  of  fry  which  lies  on  the  same  side  of 
its  axis  of  symmetry. 

Corollary:  If  2 is  a generalized  cone,  and  if  the  contours  in  its  image  decompose  in  a unique 
way  into  three  parts  that  satisfy  the  conditions  of  theorem  5,  then  those  parts  are  (i)  the 
image  of  Ty,  whose  axis  of  symmetry  is  the  image  of  the  axis  of  2;  (ii)  the  image  of  T.  the 
radial  extremities  of  2;  and  (iii)  the  image  of  •,  the  cone’s  fluting. 

Once  again  this  result  enables  us  to  set  up  a system  of  constraints  on  the  contour 
present  in  an  image  such  that,  if  the  constraints  are  satisfied  by  a particular  labelling  of  the 
contour,  that  labelling  enables  us  to  discover  the  axis  A of  2,  and  other  information  about 
its  cross-section  p and  axial  scaling  function  k The  algorithms  that  implement  this  method 
need  only  recognise  the  properties  of  parallelism  and  qualitative  symmetry  between  a small 
number  of  elements.  This  result  reaches  slightly  beyond  the  scope  of  this  article  since  it 
deals  with  contour  that  is  not  necessarily  occluding.  It  also  extends  naturally  to  the  case 
where  p contains  creases  (points  of  discontinuity  in  gradient),  which  is  helpful  because 
creases  often  give  rise  to  edges  and  highlights  in  an  image. 

Nearby  timing  points  and  curvtd  axes 

The  methods  discussed  in  this  article  are  ill-suited  to  images  that  arise  from 
nearby  viewing  points,  and  are  of  little  use  for  cones  with  curved  axes  unless  their  cross- 
sections  are  simple.  These  points  are  best  made  by  figure  13,  which  shows  a serpent  weaving 
towards  and  away  from  a nearby  viewer  (figure  13a),  who  sees  an  image  that  resembles 
figure  13b.  The  points  of  inflexion  in  figure  13b  are  caused  by  perspective,  and  to  recognise 
this  one  needs  other  cues,  like  texture  gradients  and  stereopsis. 

If  one  sees  the  contour  that  appears  in  figure  13c,  one  can  and  does  infer  the  shape 
of  a snake.  Cases  such  as  figures  ISc  k d,  where  the  scaling  function  A is  roughly  constant, 
are  easy  to  deal  with;  to  are  other  cases  where  the  qualitative  symmetry  of  theorem  3 is 
reversed  (le.  convex  segments  match  conave  segments,  not  convex  ones),  but  in  general  the 
situation  can  be  complex.  I have  been  unable  to  derive  any  substantial  results  from 
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II  The  methods  of  thU  article  are  based  on  the  distinction  between  convex  and  concave 
contour  segments.  They  are  therefore  unsuitable  for  images  of  nearby  objects.  For 
example,  If  a viewer  is  close  to  a snake  (as  in  19a),  the  image  he  sees  wilt  be  something  like 
ISb.  The  convexities  and  concavities  in  this  are  mostly  due  to  the  perspective 
transformation,  and  they  do  not  reflect  properties  of  the  viewed  surface.  The  figures  in  13c 
and  d are  generalized  cones  with  curved  axea  It  is  not  known  how  to  deal  with  these  except 
in  simple  cases  like  those  depicted  here. 
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circumstances  in  which  the  surface  2 and  its  viewing  point  are  unconstrained. 


3:  Surfaces  oomposed  of  two  or  more  generalized  oones 

We  have  hitherto  been  concerned  with  the  appearance  of  a single  generalized 
cone  Real-life  objects  are  often  approximately  composed  of  several  different  cones,  joined 
together  in  various  ways  (see  Marr  8c  Nishlhara  1977  figure  6),  and  we  therefore  have  to 
study  ways  of  decomposing  a multiple  cone  into  its  components  - for  example,  a human 
body  into  arms,  legs,  torso  and  head.  The  way  in  which  two  cones  join  has  a profound  but 
usually  local  effect  on  the  contours  produced  by  the  resulting  surface,  and  it  can  upset  the 
qualitative  symmetry  and  parallelism  on  which  our  earlier  results  depended  by  interfering 
with  the  inflexion  points  on  which  primary  contour  description  is  based.  Therefore,  the 
algorithms  for  interpreting  occluding  contours  in  an  image  must  incorporate  a sensitivity  to 
situations  that  can  arise  as  a result  of  joins. 

In  this  section,  we  study  the  common  types  of  cone-cone  junction,  classify  the 
appearances  to  which  they  can  give  rise,  and  indicate  how  algorithms  for  their  detection 
may  be  constructed.  In  order  to  do  this,  we  once  again  have  to  place  some  restriction  on  the 
way  in  which  a join  is  configured.  The  one  that  I choose  is: 

Restriction  l.The  axes  of  two  pined  generalized  cones  are  coplanar, 

which  enables  one  to  relate  the  silhouette  of  the  junction  between  two  cones  to  the  angle 
between  their  axes  and  their  axial  scaling  functions.  If  the  two  axes  are  not  coplanar,  the 
surfaces  at  the  junction  are  rather  unconstrained.  In  practise,  R5  is  not  a severe  restriction. 
Provided  that  the  two  axes  approach  one  another  closely  relative  to  the  width  of  their 
respective  cones,  the  coplanar  condition  will  be  satisfied  closely  enough. 

A:  Slde-to-end  pins  between  two  generalized  cones 

The  most  useful  common  feature  of  the  join  between  two  cones  is  that  it  gives  rise 
to  one  or  two  deep  concavities  in  the  surface's  silhouette.  This  feature  is  unfortunately  not  a 
necessary  concomitant  of  a cone-cone  junction,  and  although  it  plays  a large  part  in  our 
algorithms  for  detecting  such  a junction  (Vatan  1976),  its  role  in  the  underlying  theory  is 
surprisingly  slight 

It  is  convenient  to  divide  the  types  of  join  that  can  occur  into  two  classes,  those  in 
which  the  end  of  one  cone  is  attached  to  the  side  of  the  other,  and  those  in  which  the  two 
cones  are  attached  at  their  ends.  The  two  types  of  Join  are  illustrated  in  figure  14,  and  a 
formal  statement  of  the  distinction  between  them  is  given  in  the  appendix.  These  two  cases 
arc  not  quite  exhaustive,  but  the  intermediate  cases  introduce  no  new  points  of  interest. 

From  the  point  of  view  of  diagnosing  these  joins,  the  important  difference 
between  them  is  that  there  are  often  two  concavities  associated  with  a side-end  join,  (one  on 
each  side  of  Aj  as  shown  in  figure  14a),  but  there  need  not  be  for  end-to-end  joins  (figure 
14b).  We  analyze  the  possible  configurations  case  by  case. 


contour,  due  to  the  join  itself.  Provided  that  the  cones  are  long  relative  to  their  width,  the 
total  concavity  will  be  near  180°. 
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At:  Both  corns  art  convex 

An  Important  clue  for  Joins  between  cones  is  the  existence  of  deep  concavities  in 
the  bounding  contour.  Figure  14a  illustrates  this.  Provided  that  the  end  of  one  cone  joins 
the  side  of  the  other  well  between  its  ends,  one  cannot  help  forming  substantial  concavities 
in  the  outline.  The  precise  result  that  establishes  this  for  convex  cones  is  theorem  6,  but  the 
details  may  be  confined  to  the  appendix  without  loss.  It  basically  states  that  the  total 
concavity  created  by  a Join  like  that  shown  in  figure  14a  is  nearly  180°,  and  this  gives  us  a 
method  for  detecting  such  joins.  Since  angles  of  180°  are  preserved  by  linear 
transformations,  the  effect  of  altering  the  viewing  angle  is  entirely  due  to  changes  in  the 
angles  that  are  caused  by  foreshortening  /|  or  fg  of  figure  23.  This  means  that  the  join 
will  remain  detectable  until  the  projection  of  one  cone's  length  becomes  comparable  with  the 
projection  of  its  width  (when  the  view  becomes  "unconventional”),  or  until  the  junction  is 
obscured. 

>42;  Cents  not  tvtrywhtrt  convex 

If  the  generalized  cones  Z|  and  Zg  are  not  convex  - for  example,  if  their  axial 
scaling  functions  contain  concave  segments  - the  concavities  that  "ought"  to  arise  at  their 
Junction  can  be  concealed,  appearing  as  part  of  the  concavities  due  to  their  axial  scaling 
functions.  The  simplest  case  of  this  is  shown  in  figure  15.  where  there  is  no  identifiable 
concavity  due  to  the  join.  Therefore,  although  concavities  provide  our  algorithms  with 
useful  first  places  to  look  for  joins,  we  need  somewhat  more  solid  results  on  which  to  base 
the  underlying  theory  of  join  detection. 

The  approach  we  take  for  diagnosing  joins  is  similar  to  that  of  theorems  3 to  6. 
We  establish  a set  of  constraints  that  are  satisfied  by  the  different  types  of  join,  and  argue 
that,  if  the  contours  in  an  image  decompose  into  segments  that  satisfy  these  constraints,  then 
they  may  be  interpreted  as  two  Joined  generalized  cones.  If  there  is  only  one  decomposition 
of  the  contours  that  satisfies  these  constraints,  then  the  interpretation  is  also  unique.  The 
relations  involved  are  usually  quite  simple.  We  shall  assume  that  Rf  holds  throughout  this 
section. 

Suppose  that  an  end  of  Z2  joins  the  side  of  Zi,  and  that  the  resulting  surface  is 
viewed  along  the  direction  perpendicular  to  the  planes  of  the  axes.  If  the  angle  w (figure 
14)  between  their  axes  is  small,  or  if  the  line  of  sight  lies  too  near  the  plane  of  the  axes,  only 
one  "side"  of  each  cone  may  be  visible  (figure  16).  In  such  cases,  there  are  no  symmetry 
relations  in  the  image,  and  the  cones’  axes  cannot  be  found. 

Provided  that  both  sides  of  the  cones  remain  visible  (figure  17),  convex  and 
concave  segments  that  lie  distal  to  the  join  are  uninfluenced  by  it  and  will  obey  the 
symmetry  theorem  3.  In  this  way,  the  distal  segments  of  the  cones  determine  their  axes, 
which  can  then  be  extended  back  to  the  join  (shown  dotted  in  figure  17). 

This  diagnostic  technique  relies  on  the  existence  of  segments  distal  to  the  junction, 
to  see  now  deal  with  the  case  in  which  there  are  none.  If  we  assume  that  the  join  takes 
place  entirely  within  one  segment  of  Zg,  there  are  six  possible  situations  and  they  appear  In 
the  top  and  bottom  rows  of  figure  16.  Four  arise  when  Z| and  Zg  consist  of  Just  one  segment 
each,  and  it  may  be  either  convex  or  concave,  the  other  two  arise  because  Zg  can  straddle  a 
segment  boundary  in  Z|  (column  3 of  figure  18).  It  is  convenient  to  subdivide  the  cases 
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where  Zj  Is  concave  into  those  esses  in  which  the  value  of  Aj  passes  through  a minimum 
and  then  increases  as  one  moves  dlstally  from  Z|  (bottom  row),  and  those  in  which  the 
minimum  value  of  Ag  is  achieved  at  the  distal  end  of  A|  (middle  row).  In  Figure  18,  this 
minimum  value  is  sens,  which  produces  a cusp-like  Zj.  Notice  that  case  18a  exhibits  the 
situation  described  by  theorem  8,  which  guarantees  the  presence  of  the  concave  segments 
Joining  2|  and  Zj.  for  reasonable  values  of  ^ and  wj. 

We  are  now  ready  for  the  main  result  about  side-to-end  joins.  Figure  18  explains 
what  is  happening.  In  each  of  the  cases  shown  there,  segmentation  points  P and  Q can  be 
found  that  decompose  the  contour  so  as  to  satisfy  a number  of  relationships.  Theorem  7 
defines  the  segmentation  points  precisely  by  making  these  relationships  explicit. 

Theorem  7 (Slde-to-end  Joins).  Let  Cy  be  a connected  contour  bounding  the 
image  of  two  generalised  cones  Z|  and  Zg  connected  by  a side-to-end  Join. 

We  assume  that  the  image  is  formed  from  a distant  viewpoint  chosen  such 
that  the  viewing  direction  lies  perpendicular  to  the  plane  containing  the  two 
cones’  axes  A]  and  Aj.  Assume  that  Cy  is  broken  into  segments  at  points  of 
inflexion.  Then  there  exist  two  points  P and  Q each  of  which  is  either  a 
point  of  inflexion  or  lies  within  a concave  contour  segment  such  that 
(I)  The  line  PQ  lies  within  the  figure  bounded  by  Cy 
(ii)  PQ  divides  Cy  into  two  parts  C;  and  C2.  between  the  contour  segments 
or  fragments  in  each  of  which  there  exists  a qualitative  symmetry  whose 
two  axes  are  the  images  of  A|  and  Aj 

(Ui)  P and  Q minimise  the  length  of  contour  fragment  left  unmatched  by 
these  symmetries 

(iv)  contour  fragments  in  Cj  left  unmatched  by  the  symmetry  round  A| 

would  be  matched  by  contours  whose  proximal  parts,  and  possibly  all  of 
which,  lie  in  the  interior  of  C2  0 PQ:  vlet  versa. 

(v)  the  image  of  A2  intersects  PQ  between  P and  Q 

Corollary:  If  the  points  P and  Q are  unique,  these  constraints  determine  a unique 
decomposition  of  Cy  from  which  images  of  the  two  axes  A|  may  be  recovered. 

In  practise,  it  does  not  matter  if  P and  Q are  not  unique  provided  that  an  possible  choices 
give  the  same  axes. 


B:Two  generalized  cones  joined  end-to-end 

If  Z|  or  Zj  contains  more  than  one  convex  or  concave  segment,  that  cone’s  axis 
may  be  found  for  segments  distal  to  the  Join,  Just  as  they  were  found  in  figure  17  for  side- 
to-end  Joint.  Hence  we  need  consider  only  the  cate  where  Xj  and  Zj  have  just  one  teg  went 
Once  again,  the  main  result  depends  on  characterising  the  segmentation  points  P and 
and  figure  19  gives  examples  of  segmentation  points  for  end-to-end  Joins  between  the 
various  types  of  single  segment  cone.  Theorem  8 defines  these  points  precisely;  it  is  very 
similar  to  theorem  7. 


19.  If  the  Joined  cones  are  short,  the  method  of  figure  I?  cannot  be  used.  This  figure 
illustrates  the  types  of  slde*to-end  Join  that  an  occur.  In  the  first  column,  the  left-hand 
cone  is  convex;  in  the  centre  column  it  is  conave,  and  in  the  third  column,  it  is  convex  on 
one  side  of  the  Join,  and  conave  on  the  other.  The  ocher  cone  is  convex  in  the  top  row, 
and  conave  in  the  ocher  two.  Segmentation  depends  upon  finding  the  points  P and  Q, 
which  are  defined  in  the  text  by  theorem  7 and  Illustrated  here  for  each  case. 
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Theorem  8 (End-to-end  Joins).  Let  Cy  be  a connected  contour  bounding  the 
image  of  two  generalised  cones  Z|  end  Z2  connected  by  an  end-to-end  Join. 

We  assume  that  the  image  is  formed  from  a distant  viewpoint  chosen  such 
that  the  viewing  direction  lies  perpendicular  to  the  plane  containing  the 
axes  of  the  two  cones  A|  and  A*}-  Assume  that  Cy  is  broken  into  segments 
at  points  of  inflexion.  Then  there  exist  two  points  P and  Q in  Cy  such 
that: 

(i)  Either  may  be  a point  of  inflexion,  one  (but  only  one)  may  lie  within  a 
concave  segment,  and  one  (but  only  one)  may  lie  within  a convex  segment 

(ii)  The  line  Pd  lies  within  the  figure  bounded  by  Cy 

(iii)  PQ  divides  Cy  into  two  parts  Cj  and  Cj,  between  the  contour  segments 
or  fragments  in  each  of  which  there  exists  a qualitative  symmetry  whose 
two  axes  are  the  images  of  A}  and  A2 

(iv)  P and  Q minimise  the  length  of  contour  fragment  left  unmatched  by 
these  symmetries 

(v)  contour  fragments  in  Cj  ltft  unmatched  by  the  symmetry  round  A| 
would  be  matched  by  a contour  whose  proximal  parts  at  least  lie  in  the 
interior  of  Cg  u Pd  and  via  verso. 

(vi)  the  images  of  Ai  and  A2  intersect  PQ  between  P and  Q 

Corollary:  If  the  points  P and  Q are  unique,  these  constraints  determine  a unique 
decomposition  of  Cy  from  which  images  of  the  two  axes  A|  may  be  recovered. 

Extension  to  eases  where  some  contour  segments  are  straight  Hues 
The  assumptions  R1  - R4  that  were  made  about  I]  and  ^ excluded  cases  where 
these  surfaces  contained  straight  lines.  Such  cases  are  frequent  in  real  life,  however,  and 
some  examples  are  shown  in  figures  19e  and  f.  Ids  is  a limit  of  Nd.  and  in  some  sense  else  of 
19a;  I9f  is  a limit  of  all  of  the  cases.  !9f  may  be  solved  in  the  standard  way.  Q is  the  only 
concave  point  in  the  contour,  and  it  matches  either  the  point  P,  or  it  induces  two  “nearest* 
points  P|  and  P 2 that  separate  the  two  arms  of  the  figure  from  the  rectangle  d^f^t  Both 
segmentations  are  permissible. 

Case  I9e  is  more  difficult  The  only  true  inflexion  points  are  Q)  and  Qg.  but  the 
line  Q1Q2  lies  outside  the  figure.  If  Q|  and  ty  ln  used  despite  this,  the  segmentation  10 
which  they  lead  corresponds  to  thinking  of  the  figure  a*  a rectangle  with  a piece  excised  (fit 
Holterbach  1975  p.  55).  This  would  be  the  preferred  description  if  Q]  and  ty  **•  near  W. 
Since  I9e  may  be  regarded  as  the  limit  of  I9d,  the  point  P (a  comer  Joining  two  staight  lines) 
can  be  regarded  as  a segmentation  point,  like  the  point  P in  I9d.  P then  induces  the  point 
d as  shown,  which  segments  the  figure  in  the  same  way  as  19d.  When  designing  algorithms 
for  dealing  with  cases  where  some  lines  arc  nearly  straight,  “convex”  comers  often  acquit*  a 
dual  status  that  arises  from  regarding  the  straight  lines  as  limits  of  conave  rather  than 
convex  contour  segments.  This  means  In  practise  that  straight  tines  are  somewhat  more 
difficult  to  deal  with  than  curves  since,  in  the  initial  state  of  the  algorithms  for 
implementing  the  methods  defined  here,  straight  lines  and  the  comers  to  which  they  lead 


ft  Mr  to  d exhibits  the  possible  types  of  snd-to*nd  Joins,  for  const  that  contain  only  one 
sogmont  The  segmentation  points  P and  g are  defined  by  theorem  8,  and  illustrated  here 
for  the  different  cam  They  preside  the  basis  for  interpreting  the  join  from  an  image.  19e 
It  f exhibit  the  configurations  that  arise  in  limiting  cam  where  tome  or  all  of  the  contours 
become  straight  lines.  Notice  that  in  Mf , the  symmetry  relations  have  degenerated  into 
parallelism  between  the  sides  of  each  arm  of  the  figure.  Segmentation  points  P and  Q may 
still  be  found,  although  it  is  common  for  ambiguities  to  arise  in  these  circumstances. 
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may  be  associated  with  several  possible  labellings. 

C:  Joins  between  mart  than  two  generalised  cents 
The  principal  difference  between  this  and  case  B above  Is  that  a given  point  of 
segmentation  may  have  more  than  one  match  elsewhere.  For  example,  in  the  silhouette  of 
an  octopus,  the  deep  concavity  between  each  tentacle  matches  two  others.  Also  in  this  case,  it 
is  possible  to  have  end-to-end  joins  in  which  both  P and  Q lie  within  concave  contour 
segments.  The  only  straightforward  result  about  the  case  of  multiple  joins  holds  when  all 
the  joined  axes  are  coplanar,  which  is  a common  but  restrictive  condition.  In  this  case,  the 
relevant  result  is  so  similar  to  theorems  7 and  8 that  I omit  it 


4:  Discussion 

The  purpose  of  this  article  was  to  elucidate  the  assumptions  that  can  reasonably  be 
made  when  interpreting  the  occluding  contours  in  an  image.  The  assumptions  at  which  we 
arrived  were  stated  as  restrictions  Rl  - R4,  and  it  was  then  proved  that  these  restrictions 
have  a close  relationship  to  the  assumption  that  the  viewed  surface  is  composed  of 
generalized  cones.  In  the  second  and  third  parts  of  the  article,  we  took  this  result  as  an 
assumption  and  studied  properties  of  images  of  surfaces  constructed  in  this  way.  We  found 
that  many  constraints  hold  among  portions  of  the  contours  In  images  of  such  surfaces,  and 
that  rough  symmetries  are  formed  around  the  image  of  the  cones’  axes.  The  importance  of 
these  relations  is  that  one  can  use  them  to  design  algorithms  for  finding  the  generalized 
cone-based  description  of  a contour,  and  for  extracting  any  axes  that  may  be  present  By 
applying  these  algorithms  repeatedly  to  the  contours  found  in  an  image,  one  can  often 
derive  the  S-D  model  representation  (Marr  8c  Nishihara  1977)  of  a surface’s  shape  without 
prior  knowledge  of  it  Methods  based  on  the  present  theory  will  however  fail  for  views  in 
which  one  or  more  axes  are  foreshortened  (roughly,  whenever  the  condition  lMn(x)  * d of 
page  17  is  violated). 

The  theory  presented  in  this  article  is  a pure  competence  theory,  or  a theory  at  the 
topmost  of  Marr  It  Poggio’s  (1976)  four  levels.  It  is  concerned  with  ends  not  means.  The 
natural  division  between  means  and  ends  is  interestingly  Illustrated  by  the  methods  for 
segmenting  a contour  into  two  component  generalized  cones  (theorems  7 It  8).  The  starting- 
point  for  our  algorithms  that  actually  find  the  points  P and  Q as  defined  by  these  theorems 
is  the  examination  of  deep  concavities  in  the  contour  Cy  (see  Vatan  1978).  This  contrasts 
strongly  with  the  theory,  because  the  concavities  may  be  small  or  even  absent,  especially  for 
end-to-end  joins.  Only  in  certain  circumstances  does  the  underlying  theory  guarantee  their 
presence  (theorem  6). 
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Appendix 

Suppose  that  Z is  an  arbitrary  three-dimensional  surface,  and  that  Sy  is  its  image 
from  viewpoint  V as  produced  by  the  projection  iy:  Z — > Sy  (figure  2 in  the  main  text). 
Sy  has  a bounding  contour  Cy  say.  that  corresponds  to  the  silhouette  of  Z.  and  which  we 
can  think  of  u having  been  obtained  by  acting  on  Sy  with  a boundary  operator  d (see 
figure  2). 

Definition.  Let  IV  be  the  set  of  points  on  Z whose  image  lies  on  Cy. 

Then  Ty  is  called  the  contour  generator  of  Cy  on  Z- 
That  is,  Ty  is  the  set  of  points  P on  Z such  that  t y(P)  lies  on  Cy.  We  can  now  define  the 
operator  |:  KZ)  • Ty  which  is  induced  by  3,  and  which  selects  Ty  out  of  Z.  This  is 
illustrated  in  figure  2.  where  3 is  defined  is  such  a way  that  the  diagram  in  figure  2 
commutes  - Le.  t yd  • dsy.  Notice  that  iy,  Cy,  Sy  and  Ty  all  depend  upon  the  vantage 
point  V. 

A formal  statement  of  the  restrictions  Rl  - RS  is  now  given: 


Restriction  Rl:  Z is  everywhere  twice  differentiable  with  continuous  second  derivative. 
Restriction  R2:  T he  inverse  %y~^ : Cy  — > Ty  is  one-valued. 

Restriction  R3:The  mapping  sy  : Ty  — > Cy  is  continuous. 

Restriction  R4:The  contour  generator  Ty  ofCy  Is  planer. 

Restriction  RJ.  The  axes  of  two  Joined  generalised  cents  art  cepianar. 

Lemma  /.  Let/fc  y)  • 0 describe  a planar  curve  that  is  twice  differentiable 
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with  continuous  second  derivative  Let  L : — > R?  be  a non-singular 

linear  transform  of  the  plane  Then  L preserves  points  of  inflexion  in  /. 

Proof:  We  define  g - If,  the  image  of /under  L,  by  |(x,  j)  y)).  Since  L is  linear, 

non-singular,  and  therefore  continuous,  it  induces  a (1  - l)  correspondence  between  the 
slopes  of  tangents  to  / and  tangents  to  g.  We  can  represent  the  set  of  possible  slopes  by  the 
unit  circle  S1.  and  so  L induces  a map  L* : 5*  — > J*.  which  is  (I  - 1)  and  onto.  Hence  L*  is 
monotonk  (either  increasing  or  decreasing)  - that  is,  if  1$  lies  between  #|  and  Ij,  and 
*r*2l  < 2v,  then  lies  between  and  L*f#2).  Now  a point  of  inflexion  X on  f 
is  a stationary  point  for  the  slope  of  the  tangents  to  /.  Because  L*  is  monotonic,  L(X)  is 
therefore  a stationary  point  for  the  slope  of  the  tangents  to  Lf  - g.  Hence  L(X)  is  an  ^ 

Inflexion  point  on  g. 

I have  used  a geometric  rather  than  an  analytic  argument  because  it  is  clear  how 
the  same  argument  applies  to  the  case  where  / is  piecewise  linear.  In  this  situation,  the 
analog  of  an  inflexion  point  is  a point  where  the  sign  of  the  change  in  gradient  reverses, 
and  the  argument  used  hen  still  applies. 

DoflnUton.  Let  p(r,  f)  • 0 be  a simple  closed  planar  curve  that  is  twice 
continuously  differentiable;  and  let  A be  a twice  continuously  differentiable 
positive  real  function.  Let  A be  a line  at  some  angle  + to  the  plane  of  p, 
and  denote  positions  along  A by  the  variable  *.  Let  Z be  the  surface  A x p. 

Then  2 is  a gtnoraUttd  amt  with  exit  A,  cross-stettem  p,  scaling  function  A, 
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and  eccentricity  f . If  f - ir/2,  X will  be  called  a r/jAr  generalized  cone.  (See 
figure  S in  the  main  text). 

Definition.  Let  S'  be  a dUtant  vantage  point  for  the  generalized  cone  X 
such  that  (i)  the  image  formed  from  V is  an  orthogonal  projection,  and  (ii) 
the  ray*  in  the  projection  all  lie  parallel  to  the  plane  of  the  crou-iection  of 
X.  Let  the  direction  of  these  ray*  in  the  plane  be  denoted  by  the  angle 
When  these  restrictions  are  in  effect,  we  shall  denote  the  contour  generator 
Ty  by  r*. 

o 

Theorem  /.  Let  X be  a generalised  cone  with  convex  cross-section  r - 0(9). 

Then  X satisfies  Rl  everywhere,  and  for  all  orthogonal  projections  parallel 
to  the  cross-section  p,  it  satisfies  the  conditions  R2  - R4.  Conversely,  if  Rl 
is  satisfied  by  the  closed  surface  X,  and  if  R2  • R4  are  satisfied  for  all 
orthogonal  projections  parallel  to  some  plane  II,  then  X Is  a generalized 
cone  with  con  rx  generating  cross-section  p that  lies  parallel  to  II. 

Proof:  Our  definition  of  a generalized  cone  ensures  that  it  satisfies  Rl.  Since  X is  generated 
by  moving  p along  the  axis  A (see  figure  5),  a given  radial  PC  - (p(9),  9)  sweeps  out  a 
plane  that  contains  A,  asp  itself  is  moved  along  A.  As  p moves,  the  radial  PC  maintains  its 
direction,  but  shrinks  or  expands  in  a manner  dictated  by  the  scaling  function  h(x).  As  O 
moves,  it  traces  out  a curve  on  X,  which  we  shall  call  Ty,  and  which  lies  in  the  plane  A. 
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Furthermore,  the  tangent  to  Z at  G that  lies  in  the  plane  of  p is  the  tangent  GV  to  p at  G, 
for  all  positions  of  G.  Suppose  that  we  represent  the  direction  of  GV  in  the  plane  of  p by  +. 
If  one  views  Z from  a great  distance  in  the  direction  A,  the  line  GV  is  a line  of  sight  to  the 
edge  of  the  surface  Z.  Therefore  G lies  on  the  contour  generator  for  this  view  of  Z.  But  this 
is  true  for  all  positions  of  PG  as  P moves  along  A.  and  so  Ty  is  a contour  generator.  In 
fact  it  is  r*.  Furthermore,  since  p is  convex,  the  same  will  be  true  for  every  angle  + and 
corresponding  point  G on  p,  provided  that  the  viewing  directions  lie  parallel  to  the  plane  of 
p.  Hence  Z satisfies  R2  - R4  for  all  such  orthogonal  projections. 

The  proof  of  the  converse  result  is  longer,  and  we  first  need  to  establish  three 

lemmas. 

Lmma  2.  Z n II*  is  convex  for  all  planes  11^  parallel  to  the  plane  II  of  the 
given  viewing  directions. 

Proof:  Suppose  that  Z n 11^  were  not  convex.  Then  there  would  exist  a line  in  11^  that  was 
tangential  to  Z n at  two  points  Gj  and  63  say,  as  shown  in  figure  20.  But  the  tine  G|G2 
is  the  ray  that  produces  the  edge  of  the  image  of  Z from  this  viewing  angle,  and  6(62 
therefore  projects  to  a point  P say,  on  Cy.  So  1 y~\P)  would  contain  both  Cj  and  G2.  and 
so  would  not  be  single* valued.  This  contradicts  R2. 

Lmma  3.  If  two  distinct  contour  generators  on  Z intersect  at  a point  X, 
then  contour  generators  for  all  distant  viewing  directions  in  the  plane  II 
pass  through  X. 

Proof:  The  tangent  plane  to  Z at  X,  which  exists  by  Rl,  contains  two  distinct  vectors  that  lie 
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In  a plan*  parallel  to  II.  Hence  the  tangent  plane  at  X mutt  itself  be  parallel  to  II. 

Ltmwta  4.  Let  and  V^2  be  contour  generators  for  two  different 
viewing  directions  in  IL  Then  and  I*^2  intersea  on  1. 

Proof.  Since  X Is  a dosed  surface,  the  I*p  for  any  angle  4 divides  X into  two  components. 
This  follows  from  the  fact  that  if  X Is  the  surface  defined  by  the  equation  fix,  j,  x)  - 0,  the 
points  on  are  solutions  to  the  equations 

Gr*d(fl . (V  - (x.  j,  t))  - 0 ( I ) 

fix,  j,t ) • 0 (2) 

where  V it  a distant  vantage  point  along  the  rays  of  the  orthogonal  projection  and  the 
two  components  correspond  to  points  where  equation  (I)  takes  values  > 0 and  < 0 respectively. 
Hence  1*^  and  V^2  each  divide  X into  two  connected  components.  Let  be  any  plane 
parallel  to  II  that  intersects  X In  more  than  a point  X n 11^  Is  a simple  closed  convex  curve, 
which  meets  in  Cj  and  Gj,  and  which  meets  in  Cj  and  G4.  as  illustrated  in  figure 
21.  The  tangents  to  X at  Cj  and  Gj  are  parallel,  and  so  are  the  tangents  at  G2  and  G4. 
Clearly,  the  line  GjGj  divides  the  simple  closed  curve  X n into  two  parts,  in  one  of  which 
lies  Gj,  and  in  the  other  of  which  UesG^ButGgandGfbothlleon  whereas  Gj  and 
63  both  lie  on  1*^  Hence  I and  V^2  intersea  somewhere  on  X. 

Conliorf:  r#;  and  intersea  twice  or  more. 

We  can  now  complete  the  proof  of  theorem  L Let  F^{  and  V^2  be  two  contour 
generators  for  X for  different  viewing  direaions  lying  In  II.  Since  and  V+2  are  both 
planar  (by  R4j,  their  containing  planet  inters mx  in  a line  A (say).  By  lemma  4.  and 


r*2  Intersect  in  at  least  two  points,  and  these  points  must  therefore  lie  alon|  the  line  A.  Let 
N be  a boundary  point  of  the  set  of  intersection  paints  of  1*^  and  I*^  on  A,  and  let  5 be 
the  next  closest  such  point  to  N.  This  situation  is  depicted  in  figure  22.  By  lemma  2,  all 
contour  generators  pass  through  N and  5,  which  we  may  therefore  think  of  as  north  and 
south  poles  of  2,  and  therefore  the  planes  of  all  countour  generators  for  views  of  2 from  II 
must  contain  N and  5 and  hence  the  line  A-  That  is,  the  planes  of  all  contour  generators  for 
distant  views  from  the  plane  II  will  intersect  in  A. 

Let  R1  be  a plane  parallel  to  II  lying  between  N and  5,  distant  s from  N,  as 
shown  in  figure  22a.  Let  11^  intersect  A at  H,  at  G\,  and  at  c2 • The 
configuration  in  11^  is  shown  in  figure  22b.  The  crucial  step  in  the  proof  is  to  notice  that 
up  to  scalar  magnification,  the  geometry  of  figure  22b  is  independent  of  z,  the  position  of 
II'  along  the  line  NS.  This  follows  from  the  following  observations: 

(i)  The  angle  between  HG\  and  HG%  is  independent  of  z,  because  it  is  simply  the  angle 
between  the  planes  of  T^j  and  measured  parallel  to  II; 

(ii)  The  direction  of  the  tangent  to  p at  0|  is  independent  of  z,  because  as  z increases,  Gj 
traces  out  the  contour  generator  P^,  which  is  by  definition  the  locus  of  tangents  to  2 
parallel  to  II  for  a given  fixed  viewing  direction 

We  deduce  that  for  each  angle  I in  figure  22b,  the  tangent  to  the  curve  p has  a 
constant  direction  for  every  z.  That  Is,  for  each  z,  the  cross-sections  p of  2 in  11^  are  all 
solutions  of  the  sam  equation 

rd$tdr  . fip)  (l) 

where/  is  tome  function  of  the  viewing  angle  A and  is  independent  of  z.  Lot  R(9)  be  a 
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mIuUor  to  (IX  Then  the  cross  section  function  p of  2 has  the  form 

Ml.  t)  - Mz)Jl(l)  (2) 

where  A is  a positive  real  function  of  a FinaRy,  Z is  twice  continuously  differentiable,  by  Rt. 
Hence  A is  a twice  continuously  differentiable  function  ofz.andioUAof#.  This  completes 
the  proof  of  theorem  L 
Corollary  J;  A - 0 at  N and  at  S. 

Corollary  J:  The  cross-section  0 can  change  at  the  poles  where  A - 0. 

Tkmm  2.  A necessary  and  sufficient  condition  for  Z to  satisfy  Rl,  and 
R2-  R4  far  all  distant  viewing  positions,  is  that  2 be  a quadratic  surface. 

Rroof.’U  2 to  a quadratic  surface;  it  satisfies  Rl  - R4.  This  follows  from  the  following  three 
obeervatlona: 

(i)  A sphere  trivially  satisfies  Rl  - R4 

(to)  Any  linear  transformation  or  transhtion  preserves  Rl  - R4 

(lit)  Any  quadratic  surface  may  be  derived  from  a sphere  by  a linear  transformation  and  a 
translation. 

Conversely,  suppose  that  2 satisfies  Rl  - R4  tor  al  viewing  angles.  Since  the 
conditions  of  theorem  I hold  for  every  viewing  angle,  2 may  be  thought  of  as  a generalized 
cone  with  generating  cross-sections  in  any  direction.  Hence,  any  two  paraM  planes  intersect 
2 in  curves  that,  if  not  null,  have  the  same  shape.  Suppose  that  2 to  the  surface  represented 
by  the  polynomial /fx,  y,  x)  - 0.  Then  a sat  of  such  paraM  planes  is  given  by  the  family 
x-ex*ty«c.for  varying  c.  We  deduce  that  the  curves 
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fix.  y.  ax  * by  * ej)  - 0 
fix,  % ax*  by*  cj)  • 0 

are  identical,  up  to  magnification  and  a translation.  Hence  the  Ct  cannot  multiply  terms  of 
second  or  higher  order.  Therefore,  in  the  original  equation,  t cannot  multiply  terms  of 
second  or  higher  order.  Since  the  condition  holds  for  arbitrary  planes,  it  mutt  also  be  true 
of  x and  of  y,  and  it  must  hold  identically.  Hence / is  at  most  quadratic,  and  we  have 
already  seen  that  quadratic  surfaces  satisfy  the  conditions  of  the  theorem. 

Romarks  about  tht  proofs  of  therms  l and  2 

The  premises  of  these  theorems,  and  the  tray  the  premises  are  used  in  the  proofs, 
seem  generally  reasonable  with  one  possible  exception.  That  U tha  uta  of  R2  to  show  that 
2 n is  convex  in  lemma  2.  In  fact,  this  is  how  R2  forces  the  cross-section  of  2 to  be 
convex,  which  is  a condition  that  holds  all  through  these  results.  One  might  argue  that  this 
is  a somewhat  artificial  use  of  R2,  which  was  introduced  mainly  to  exclude  surfaces  that 
contain  lines.  The  analysis  given  so  far  concerns  only  the  silhouette  of  the  image  of  2 
however,  and  in  these  circumstances  the  convexity  assumption  for  p is  a reasonable  one, 
because  violations  cannot  be  detected  from  viewing  directions  that  He  parallel  to  the  plane 
of  0.  In  practise,  one  can  apply  the  above  analysis  to  all  occluding  contours  in  an  image, 
provided  that  one  subsequently  relates  the  different  cylinder  descriptions  that  emerge  for 
different  parti  of  the  same  surface.  Figure  I shows  a simple  example  of  this. 


T hoar  oat  f (Antal  Symmotry).  Let2-pxAbea  generalised  cone  with 
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namely  c},  but  in  general  there  may  be  2.  Hence  the  symmetry  established  by  the  axis  A* 
holds  for  at  least  (n  - 2)  of  C^'t  contour  segments,  (or  2 of  them  If  n - 3),  and  it  matches 
up  all  contour  segments  that  are  concave,  since  the  end  segments  must  be  convex.  Finally, 
there  must  be  at  least  three  contour  segments,  since  the  premises  of  the  theorem  require  that 
there  be  2 concave  ones,  and  is  a simple  closed  curve. 


Definition.  The  type  of  symmetry  established  by  theorem  3,  which  holds 
between  convex  or  conave  segments  of  a contour  and  which  includes  a 
sating  factor,  we  shall  all  a qualitative  symmetry. 


Definition.  The  set  T of  points  for  all  # and  each  value  of  zi  that 

makes  Mzt)  a teal  maximum  or  minimum,  is  ailed  the  radial  extremity  of 
2.  Let  V be  an  arbitrary  distant  vantage  point,  and  let  vl  be  the  projection 
from  V onto  a cross-section  plane  of  2,  as  described  in  the  main  text  and 
illustrated  in  figure  9.  The  set  Tyl  o T is  ailed  the  skeleton  of  2 far  the 
vantage  point  V. 

Theorem  4 (Skeleton  Theorem).  Let  Tyl  u T be  the  skeleton  of  2 associated 
with  some  distant  vantage  point  V.  Then  provided  that  Cy  is  the  projection 
of  Tyl  to  V, 

(l)  Cy  is  qualitatively  symmetric  about  the  image  oftheaxisAof2.inthe 
sense  of  theorem  I 
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(U)  the  lim(t  of  T consists  of  one  or  more  connected  components,  through 
which  A posses,  end  between  any  two  of  which  there  exists  e mapping  that 
is  (l  - !),  continuous  and  onto,  that  preserves  the  gradient  of  the  image  of  T 
at  each  point 

Protf:  V correponds  to  a viewing  direction  that  is  coptanar  with  the  cross-section  p.  Hence 
the  contours  Cyl  in  the  image  as  seen  from  W obey  the  conditions  of  theorem  3,  end  the 
image  of  the  axis  A induces  a qualitative  symmetry  between  its  concave  and  convex 
components.  Such  relations  are  preserved  by  an  orthogonal  projection,  and  since  it  is  a 
condition  of  the  theorem  that  Cy  coincides  with  the  projection  of  Tyl.  it  follows  that  Cy  is 
also  qualitatively  symmetric  about  the  image  of  A.  Secondly,  T trivially  consists  of  one  or 
more  connected  components  through  which  A passes,  since  these  components  are  Just  cross* 
sections  at  the  local  maxima  and  minima  of  A Finally,  if  Ht\)  and  are  local  maxima 
or  minima  of  A the  mapping  between  points  on  Z given  by 
(#*!>*(«. #.  ij)  — > f . *j> 

is  continuous,  (I  * I),  onto,  and  preserves  the  gradient  of  T at  each  point,  since  the  gradient 
of  both  at  f is  pdpldt.  This  correspondence  is  preserved  by  an  orthogonal  projection,  and 
hence  the  relations  will  still  hold  in  the  image  of  T. 


Definition.  Let  Z be  a generalised  cone,  and  let  • be  the  set  of  points 
(k(x)+($l).  #j,  xj  for  all  x,  for  each  #t  that  makes  p a local  maximum  or 
minimum.  Then  # is  called  the/httaf  of  Z.  For  a viewpoint  K,  let  Tyl  u 
T bo  the  skeleton  of  Z that  occurs  in  theorem  1 We  define  the  complete 
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sMtto n by  adding  the  fluting  to  7i  skeleton,  la.  the  complete  skeleton  of  X 
from  viewpoint  V is  the  set  Ty*  o T u #. 

Tktorrm  5.  Let  Ty^  u T u * be  the  complete  skeleton  of  X associated  with 

some  distant  vantage  point  V.  Then 

(i)  Cy  and  the  image  of  T obey  theorem  4 

<ii)  The  image  of  each  portion  [(Hx)^),  #4,  x),  for  fixed  #|  and  varying 
x)  of  the  fluting  of  X is  either  a straight  line,  or  it  divides  into  convex  and 
concave  segments  that  are  in  (/  * l)  correspondence  with  the  convexities  and 
concavities  in  that  part  of  Cy  which  lies  on  the  same  side  of  in  axis  of 
symmetry. 

Proof:  Part  (i)  follows  from  theorem  4.  Part  (ii)  follows  because  like  the  two  qualitatively 
symmetric  components  of  Cy,  each  contour  in  the  fluting  of  X is  generated  by  variations  in 
h(t).  If  such  a contour  lies  directly  on  the  line  of  sight  to  the  axis  A of  X.  it  win  appear  in 
the  image  as  a straight  line  Otherwise  its  concavities  and  convexities  will  follow  those  of 
one  component  of  Cy,  although  the  depth  of  the  concavities  or  convexities  will  differ  in 
general 


DofinUtem.  For  ! - /,  2 let  X|  be  a generalised  cone  with  maximum  width 
2»t  and  axis  A|  of  length  /j.  Let  Xj  and  Xg  be  Joined,  and  let  w be  the 


angle  between  their  axes  (see  figures  14  and  23).  Then  the  Join  between  the 
cones  wIN  be  called  stiol+ond  (the  side  of  X|  to  the  end  of  Xg)  provided 
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) 

that 

(i)  the  two  axes  intersect  between  the  ends  of  A| 

(ii)  the  whole  of  the  Joined  end  AB  of  2g  lies  between  the  two  lines 
perpendicular  to  and  passing  through  the  ends  of  A| 

(lit)  one  of  the  points  A and  B of  figure  14  does  not  Ue  within  the  convex 
hud  of  2|. 

The  Join  is  called  tnd-to-tnd  if 

(i)  the  two  nearest  ends  of  A|  and  A2  ere  within  min  (wj,  m2)  of  one 
another,  and 

(11)  the  two  furthest  ends  are  greater  than  (wj  ♦ m2)  apart 


Thtorm  6.  Let  Z(  and  Zg  be  two  convex  generalised  cones  such  that  the 
end  of  Zg  Joins  the  side  of  Zj.  and  the  Join  satisfies  restriction  M.  Let  the 
lengths  of  the  cones’  axes  A|  be  /j.  and  let  the  diameters  of  their  crow- 
sections  be  bounded  by  2ml  (l  • I,  2).  Then 

(I)  the  only  concavities  that  can  occur  in  the  image  are  due  to  the  Junction 
(ii)  viewed  distantly  perpendicular  to  the  plane  of  A|  and  Ag,  the  total 
concave  angle  present  in  the  image  is  near  /W°  (in  the  tense  made  precise 
in  the  proof)  provided  that  the  site  of  the  Join  it  not  near  an  end  of  Zj, 
and  that  2]  and  Z2  ere  much  longer  than  they  are  wide. 

Fmf:  By  lemma  b,  all  contours  derived  separately  from  Z|  and  Zg  are  convex.  Hence  any 
concavity  in  the  image  of  their  union  mutt  be  due  to  the  way  they  are  Joined.  If  Z|  and  Zg 
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have  coplanar  axes  and  are  viewed  perpendicularly  to  this  plane,  the  resulting  configuration 
is  as  represented  in  figure  23.  The  contours  shown  in  thick  lines  there  represent  cylinders  /j 
long  and  V|  thick  which,  by  the  conditions  of  the  theorem,  bound  the  cones  Zj  (t  - I,  2).  Let 
P be  one  of  the  two  points  at  which  contours  due  to  Z|  and  Zj  intersect,  and  let  PQ^  and 
PQ.2  **  th«  tangents  to  Z|  and  Zj  at  P-  Let  m be  the  angle  between  the  axes  A|  and  A2  of 
Z]  and  Z2.  and  let  fa  and  ^ be  the  angles  that  PQ\  and  Pfa  make  with  Aj  and  A^  Then 
the  angle  between  PQj  and  PQj  is  « ■ (180°  • « - ^ - fa).  The  corresponding  angle  at  the 
other  intersection  pf  between  contours  of  Z]  and  fa  it  - (m  - fa  - +4),  as  illustrated  in 
figure  23.  Hence  the  total  concavity  due  to  the  Join  is  a * J - ( 180 0 • fa-  fa  - fa-  +4)- 
In  order  to  establish  a lower  bound  for  the  total  concavity,  we  need  to  find  upper  bounds 
for  the  angles  fa  (t  - / to  4),  and  we  can  use  the  convexity  of  Z|  and  fa  to  do  this.  Since 
the  scaling  functions  of  Z|  and  Z2  »re  convex,  the  maximum  possible  value  of  fa  fa 
(shown  in  figure  23),  which  is  exactly  - wyuxfu)),  and  approximately 

tari’iffalty-  The  maximum  possible  value  of  fa  is  d]>  which  is  approximately  tan'k a»|/rf|) 
(see  figure  23>,  and  similarly  for  fa  and  fa  These  approximations  hold  provided  that  the 
cones  are  long  relative  to  their  widths,  and  dt  is  not  near  0 or  - Le.  the  Join  is  not  near 
either  end  of  Z]. 

Theorems  7 and  8 were  stated  precisely  in  the  text  and  their  proofs,  which  are 


straightforward,  are  omitted. 


# 


BLOCK  20  CONTINUED: 

arise  from  contiguous  parts  of  the  viewed  surface— i.e. , there  are  no  invisible 
obscuring  edges.  It  is  proved  that,  for  smooth  surfaces,  these  assumptions 
are  essentially  equivalent  to  assuming  that  the  viewed  surface  Is  a generalized 
cone.  Methods  are  defined  for  finding  the  axis  of  such  a cone,  and  for 
segmenting  a surface  constructed  of  several  cones  into  its  components,  whose 
axes  can  then  be  found  separately.  These  methods,  together  with  the  algorithms 
for  implementing  them  devised  by  Vatan  & Marr  (1977),  provide  one  link  between 
an  uninterpreted  figure  extracted  from  an  image,  and  the  3-D  representation 
theory  of  Marr  & Nishlhara  (1977). 
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